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Abstract 

We present an algorithm testing wheather, for given four vectors in 
R^, there is a plane through the origin such that all four vectors fall into 
the same open halfspace. 

The following problem was communicated to me by Igor Komarov. Suppose 
we are given four vectors xi, . . . ,X4 G M'^. How to test if there is a supporting 
plane, that is, a plane P such that all four vectors lie in the same open halfspace 
with respect to this plane? (When we talk about planes or lines, we mean linear 
subspaces, that is, containing the origin.) 

This was needed for Komarov's survey paper the carbon atoms in a 
molecule are classified there into two types depending on the existence of a 
supporting plane (with respect to the four vectors corresponding to the four 
ties at the carbon atom). 

We did not find an explicit algorithm in the literature and, in order to keep Q 
short, we wrote the present note with the description of the algorithm and the 
proof of its correctness. 

Let each Xi be represented by its Cartesian coordinates, viewed as a column 
vector: Xi = {xii,Xi2,Xi3)'^ . Clearly, the existence a supporting plane is equiv- 
alent to the existence of a linear function f{y) = J2^=i fnn ^ith /(^O > for 
all 1 < « < 4. 

It is obvious how to extend the above notions to the general case of k vectors 
in W^. Here is a criterion that we will use. 

Lemma 1 Let xi, . . . ,Xk G M'*. A supporting {d — 1)- dimensional hyperplane 
does not exist if and only if there are weights Wi > 0, 1 < i < k, not all zero, 
with XliLl "^i^i = 0- 
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Proof. If a supporting plane does not exist, then the system of hnear inequahties 
fi^ij ^ 1 has no solution in the /'s. Now the existence of weights is pre- 
cisely the conclusion of the Farkas Lemma. (For example, apply Proposition 1.7 
from 1^ with Aij = —Xij and = — 1, 1 < i < fc, 1 < j < d.) 

On the other hand, if we have weights, then for any / we have X^iLi 'Wif{xi) = 
fiJ2i=i ^i^*) = 0' so f{xi) < for some i. | 

Return to our xi, . . . ,X4 £ M'^. First, we handle the case when some three 
vectors are coplanar. 

Theorem 2 Let xi, X2 and xs lie on a plane P. There exists a supporting 
plane if and only if the set {xi, . . . jX^} n P admits a supporting line in P. 

Proof. If X4 £ P, then both weight conditions of Lemma |l| are identical. If 
X4 ^ P, then a supporting plane does not exist iff ''^i^i — fo'' some 

weights, which is the case iff X^i'^i '^i^^i = for some weights because X4 does 
not lie in the linear span of xi , X2 , . I 

Testing in a plane is probably done easiest through the following observation: 
a supporting line for vectors , . . . , Xfc £ exists iff there is a relabelling of 
subscripts such that Z{xi,Xk) — Yli=i '^i'^ij'^i+i) ^^'^ ^{xi,Xk) < tt, where 
< Z(a;, < TT is the angle between vectors x and y. 

It remains to consider the case when no three vectors are coplanar. Let 
P{xi , Xj ) be the plane passing through Xi and Xj . Here is a corollary to Lemma ^ 

Corollary 3 A supporting plane exists if and only if for some 1 < « < j < 4, 
the remaining two vectors lie in the same open halfspace with respect to the plane 

P(,X^ , Xj ) . 

Proof. Suppose we have a supporting plane P. We can rotate P until it hits 
some Xi. If P does not contain other a;'s, rotate it further, along the axis Oxi, 
until it hits for the first time some Xj. Now, P{xi,Xj) is clearly the required 
plane. 

Conversely, suppose that a supporting plane does not exist, that is, by 
Lemma |l| we can find weights. Let / = be the equation of the plane P{xi,X2). 
We have /(S3) ^ and /(X4) ^ and the equahty J2h=i ^hf(xh) = imphes 
that fixz) ■ f{x4) < 0, as required. | 

So, the algorithm would be to take all six pairs 1 < i < j < 4, write an 
equation of the plane P(xi,Xj), which is 

f{y) = det{xi,Xj,y) ^ 0, 

and check the signs of / on the remaining two vectors. However, this procedure is 
not very economical as the determinant of a matrix, say (xi, X2, xa), is computed 
essentially three times. Here is a better algorithm. 
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Theorem 4 Compute the signs of 

det(S'i, X2, xs), det (xi , X4 , X2 ) , det(x3, ^2, 2:4), det(xi, X3, X4). 
A supporting plane does not exist if and only if all four signs are the same. 

Proof. Suppose that a supporting plane does not exist. Assume det(S'i,a;2,5;3) > 
0. Then applying Corollary ^ to P{xi,X2) we obtain that det(a:i,X2, X4) < 
0, that is, det(a;i, X4, a;2) > as required. Similarly, det(x3, a;2, X4) > and 
det(xi, X3, a;4) > 0. 

The converse also follows by an easy application of Corollary ^ e.g. X3 and 
Xi are separated by P(xi,X2) because 

det(xi, X2, 0:3) • det(a;i, a;2, a;4) = — det(xi, ^2, X3) • det(a;i, X4, 5:2) < 0. | 
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